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NOTE  ON  B.  KLEIN'S  "DIRECT  USE  OF  EXTREMAL  PRINCIPLES 
IN  SOLVING  CERTAIN  PROBLEMS  INVOLVING  INEQUALITIES” 

1 .  INTRODUCTION 

In  the  May,  I •  issue  of  the  Journal  of  the  Operations 
Research  Society  of  America,  B.  Klein  [l]  proposed  that  ordinary 
methodo  of  the  differential  calculus  be  used  to  minimize  a 
function  z  of  n  variables  ,x2 , . . .  ,xn,  where  the  latter  are 
subject  to  inequality  constraints  instead  of  the  usual  equality 
constraints.  Indeed,  as  a  first  step,  by  the  introduction  of 
new  variables  he  replaces  all  inequalities  such  as 

(1)  f1(x1,x2,...,xn)  £  0 

by  equ  ill ties  such  as 

U)  f1(x1,x2, . .  .,xn)  +  u*  -  0. 

After  this  the  problem  can  be  solved  (in  theory)  by  the  use  of 
Lagrange  multipliers.  For  the  linear  programming  case  the 
method  yields  the  information  that  the  solution  is  at  a  vertex 
where  the  sign  of  equality  holds  in  some  of  the  original  con¬ 
straints  but  perhaps  not  in  others.  For  the  nonlinear  case  it 
shows  in  general  only  that  the  solution  is  either  inside  or  on 
the  boundary*  Our  purpose  will  be  to  discuss  whether  this 
method  of  analysis  can  be  used  constructively  to  determine  the 
optimum. 
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2.  APPLICATION  TO  UNEAR  PROGRAMMING  PROBLEMS 

Por  the  standard  linear  programing  problem 
n 

(3)  **  (^  **  ^  #  2 »  •  •  •  *rn ) » 

(4)  x ,  >  0  (J*l#2,...,n), 

n 

(5)  £  c.x,  »  z  (minimum) 

J-l  J  J 

the  inequality  conditions  (4)  are  replaced  by 

(4‘)  -  Xj  ♦  u*  «  0  (J  -  1,2, 

Let  the  Lagrange  multipliers  associated  with  the  1-th  e;uatlcn 
of  (})  and  the  J-th  equation  of  (4')  be  and  5^,  respectively. 
Then  the  main  problem  is  reduced  to  that  of  finding  the  uncon¬ 
strained  minimum  of  the  function 

(6)  «•  -  Ji  CjXj  -  £  -ijXj  -  bt)  ♦  Ji  &j(-Xj  ♦  - 

where  the  constants  v,  and  5,  are  selected  3u  that  x<  and  u. 

i  O  J  J 

at  the  minimum  satisfy  (^)  and  (4').  Setting  dz'/ax,  «*  0  and 

J 

dz'/au,  *  0  yields  necessary  conditions  for  a  minimum: 

J 

m 

(7)  c  j  ~  X 


(8) 


Vj  *  °> 


or — eolving  fur  5j  In  (7)  and  substituting  in  (8) 

(J)  (CJ  *  it,  ’iao)  UJ  ■ 


0 


(J  •  1,2, ...,n). 
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which  are  well-known  relations  that  must  exist  between  optimal 
solutions  ol'  the  primal  and  dual  linear  programming  problems. 

The  essence  of  Klein's  proposal  is  that  (9)  implies  for 
each  J  that  either  the  first  or  the  second  factor  must  be  zero. 
This  gives  rise  to  2n  possible  ways  that  (9)  can  be  satisfied. 
Consider,  for  Instance,  the  possibility  that  for 
the  first  factjr  of  the  product  vanishes  and  for  all  other  J 
the  second  vanishes.  The  n  —  r  conditions  Uj  •  0  (i.e.,  Xj  -  0) 
for  J  +  when  substituted  into  (})  give  rise  to  m  equations 

(10) ,  below,  in  r  variables  Xj  where  J  •  ,  and  the  r  con¬ 

ditions  5.-0  for  J  -  J  give  rise  to  r  equations  (11)  in  m 

j  ^ 

variables  ir^: 

r 

(10)  £  a.  .  x.  —  b.  (l  —  1 ,2, . . . ,m), 

*-l  XV  1 

m 

(1*)  Z  *iai  j  ■  cj  i* m  1*2, ...,r)  . 

If  r  <  m,  this  would  result  In  the  first  system  (10)  having 
fewer  equations  than  variables;  If  r  >  m,  the  second  system 

(11 ) ,  whose  matrix  Is  the  transpose  of  the  first,  would  have 
this  property.  Thus  in  practice  it  may  be  expected  that  each 
member  of  a  large  subset  of  the  2n  combinations  will  result  in 
an  inconsistent  system.  For  example,  if  one  could  assume  that 
the  ran*  of  any  m  x  m  submatrix  of  the  coefficients  of  (1)  is  m, 
then  all  cases  r  >  m  or  r  <  m  could  be  dropped;  thus  Instead 

of  having  to  handle  2n  combinations,  one  could  deal  with  the 
(Jj)  eomblnatlona  where  r  -  m.  To  avoid  the  practical  difficulty 


of  testing  the  assumption  on  rank,  one  could  Introduce  "arti¬ 
ficial"  variables  as  is  usually  done  in  linear  programming  and 
show  that  it  is  sufficient  to  consider  cases  r  -  m  with  sub- 
matrices  of  rank  m. 

In  the  case  of  a  linear  programming  problem  for  which  values 
xj  exist  satisfying  (3)  and  (4)  and  for  which  there  is  a  finite 
lower  bound  to  the  value  of  z,  the  procedure  as  outlined  is 
completely  valid.  One  simply  selects,  from  among  the  combina¬ 
tions,  where  n-m  of  the  x«  are  set  equal  to  zero,  solutions  In 
which  all  Xj  satisfy  Xj  £  0  and  finds  one  that  minimizes  the 
value  of  z. 

3.  COMPUTATIONAL  CONSIDERATIONS 

It  has  been  observed  that  many  people  who  have  considered 
problems  involving  inequalities  have  gone  over  the  same  ground 
as  Klein,  which  we  have  Just  discussed.  The  practical  dlfflcu  iSJL 
arises  only  if  the  solution  of  (” )  systems,  each  m  x  m,  is  too 
much  work.  Thus  if  n  ■  7,  m  »  2  as  in  the  illustrative  example 
presented  by  Klein,  all  combinations  can  be  enumerated  quickly. 
Oeorge  Stlgler  [2],  on  the  other  hand,  In  trying  to  solve  a 
nutrition  problem  involving  77  food3  and  9  nutritional  elements, 
made  a  search  among  the  combinations  of  an  (n  »  77,  m  ■  9)  linear  — 
programming  nutrition  problem.  Actually,  he  was  able  tu  eliminate 
many  foods  and  needed  to  consider  an  (n  ■  23,  m  -  9)  case;  since 
(  (r)  was  also  quite  large,  he  had  to  resort  to  further  devices 
to  decrease  the  number  of  com.blnatiwn3  to  ib  ut  ‘.12.  Ev^n  after 
this  reduction,  since  this  was  done  in  the  days  of  hand  methods, 
he  was  forced  to  try  a  few  of  the  remaining  combinations  and  take 
for  his  "optimum"  the  best  among  them. 


4  .  THE  SIMPLEX  METHOD 


The  simplex  method  [3]  In  fact  begins  where  Klein  leaves 

off.  The  first  part  of  the  procedure  consists  of  a  method  of 

selecting,  from  among  the  (J[)  combinations,  a  solution— called 

a  basic  feasible  solution— In  which  all  Xj  satisfy  Xj  >  0  (the 

m  variables  x,  ,  x,  ,  ...,x.  arc  called  basic  variables).  With 

such  a  starting  solution,  the  values  of  the  are  determined 

next  by  (11)  and  of  the  6^  by  (7).  It  can  be  shown  that  all  6j 

should  satisfy  6  £  Q  for  the  solution  to  be  optimum.  Thus,  If 

there  Is  a  8#  <0,  one  can  allow  the  corresponding  nonbaslc 

variable  x  ■  0  to  Increase  while  all  other  nonbaslc  variables 

remain  at  value  zero.  At  a  value  x#  •  xj  sufficiently  large  one 

of  the  values  of  the  basic  variables  becomes  x.  -  xf  *0  while 

*o  xo 

all  other  x.  satisfy  x.  >0.  This  gives  rise  to  a  second 

- 

combination  In  which  x_  replaces  x.  as  basic  variable.  The 

3  xo 

new  value  of  z  can  be  shown  to  be  less  than  that  for  the  previous 
combination.  The  procedure  is  Iterated  until  a  solution  satis¬ 
fying  (3)  and  (4)  is  obtained  in  which  all  6j  satisfy  Sj  £  0# 
Approximately  m  combinations  are  usually  examined  In  practice. 

The  shift  from  one  combination  to  the  next  can  be  arranged  so 
that  no  more  than  inn  new  multiplications  are  needed.  Thus 
each  new  combination  does  not  Involve  solving  from  scratch  a 
new  m  x  m  system  of  equations . 

- 1 - 

The  values  of  the  n-m  nonbaslc  variables  ars  zero. 
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